Why agents for automated negotiations should be adaptive

We show that adaptive agents on the Internet can learn to exploit bidding agents who use a (limited) number of fixed strategies. These learning agents can be generated by adapting a special kind of finite automata with evolutionary algorithms (EAs). Our approach is especially powerful if the adaptive agent participates in frequently occurring micro-transactions, where there is sufficient opportunity for the agent to learn online from past negotiations. More in general, results presented in this paper provide a solid basis for the further development of adaptive agents for Internet applications

Co-evolving automata negotiate with a variety of opponents

Real-life negotiations typically involve multiple parties with different preferences for the different issues and bargaining strategies which change over time. Such a dynamic environment (with imperfect information) is addressed in this paper with a multi-population evolutionary algorithm (EA). Each population represents an evolving collection of bargaining strategies in our setup. The bargaining strategies are represented by a special kind of finite automata, which require only two transitions per state. We show that such automata (with a limited complexity) are a suitable choice in a computational setting. We furthermore describe an EA which generates highly-efficient bargaining automata in the course of time. A series of computational experiments shows that co-evolving automata are able to discriminate successfully between different opponents, although they receive no explicit information about the identity or preferences of their opponents. These results are important for the further development of evolving automata for real-life (agent system) applications

Why agents for automated negotiations should be adaptive

We show that adaptive agents on the Internet can learn to exploit bidding agents who use a (limited) number of fixed strategies. These learning agents can be generated by adapting a special kind of finite automata with evolutionary algorithms (EAs). Our approach is especially powerful if the adaptive agent participates in frequently occurring micro-transactions, where there is sufficient opportunity for the agent to learn online from past negotiations. More in general, results presented in this paper provide a solid basis for the further development of adaptive agents for Internet applications

Equilibrium selection in alternating-offers bargaining models : the evolutionary computing approach

This paper studies the dynamic and equilibrium-selecting behavior of a multi-agent system consisting of adaptive bargaining agents. We model an adaptive agent as a collection of strategies which is optimized by an evolutionary algorithm (EA). EAs are stochastic search methods based upon the principles of natural genetic systems. These algorithms have been used in the past, with considerable success, to solve difficult optimization problems. Examples include problems with huge search spaces, multiple local optima, discontinuities, and noise. Adaptive agents learn in three different ways in an evolutionary setting: (i) by selection and reproduction of successful strategies, (ii) by recombining or ``crossing over'' previously-tested strategies, and (iii) by random experimentation (by ``mutating'' existing strategies). Such agents are boundedly rational because they only experience the profit of their interactions with other agents and learn by trial-and-error instead of abstract reasoning. Their equilibrium-selecting behavior is interpreted in this paper by comparison with game-theoretic (subgame-perfect equilibrium) predictions for fully rational agents. This paper shows that game-theoretic approaches are very useful to interpret equilibrium-selecting behavior in evolutionary systems of adaptive bargaining agents. The adaptive agents are boundedly rational because they only experience the profit of their interactions with other agents. Nevertheless, they display behavior that is surprisingly "rational" and fully informed in many instances. Agreement between theory and experiment is especially good when the agents experience an intermediate time pressure. In extreme situations (i.e., when time pressure becomes either strong or weak) significant deviations from game-theoretic predictions can occur, however. A good example is the case of extreme time pressure. In this case, highly nonlinear transients can occur if the deal reached by the adaptive agents approaches the extreme outcome predicted by game theory. Two other experimental observations should also be mentioned here. First, the finite horizon of the negotiations is not always fully exploited by the last agent in turn (even if time pressure is rather weak). In fact, the boundedly-rational agents often act as if the length of the game is actually much longer. This lends more support to the "infinite-horizon" assumption frequently employed in game-theoretic work. This approximation may yield surprisingly accurate results when the agents do not perceive the deadline of the negotiations. Second, we observe (and explain) discrepancies between theory and experiment if the agents experience an unequal time pressure. More in general, this work presents a systematic validation of evolutionary and computational techniques in the field of bargaining. Our model has also served as a starting point for further explorations. Several important topics have been addressed in these works: complex multi-issue and multi-opponent bargaining problems, economic modelling issues, learning by co-evolution, the development of powerful bargaining strategies, etc. We hope that these different lines of research will be extended further in future works

The influence of evolutionary selection schemes on the iterated prisoner's dilemma

Many economic and social systems are essentially large multi-agent systems. By means of computational modeling, the complicated behavior of such systems can be investigated. Modeling a multi-agent system as an evolutionary agent system, several important choices have to be made for evolutionary operators. Especially, it is to be expected that evolutionary dynamics substantially depend on the selection scheme. We therefore investigate the influence of evolutionary selection mechanisms on a fundamental problem: the iterated prisoner's dilemma (IPD), an elegant model for the emergence of cooperation in a multi-agent system. We observe various types of behavior, cooperation level, and stability, depending on the selection mechanism and the selection intensity. Hence, our results are important for (1) The proper choice and application of election schemes when modeling real economic situations and (2) assessing the validity of the conclusions drawn from computer experiments with these models. We also conclude that the role of selection in the evolution of multi-agent systems should be investigated further, for instance using more detailed and complex agent interaction models

Equilibrium selection in alternating-offers bargaining models: the evolutionary computing approach

A systematic validation of evolutionary techniques in the field of bargaining is presented. For this purpose, the dynamic and equilibrium-selecting behavior of a multi-agent system consisting of adaptive bargaining agents is investigated. The agents' bargaining strategies are updated by an evolutionary algorithm (EA), an innovative computational method to simulate collective learnin g in societies of boundedly-rational agents. Negotiations between the agents are governed by the well-known``alternating-offers' protocol. Using this protocol, the influence of various important factors (like the finite length of the game, time preferences, exogenous breakdown, and risk aversiveness) is investigated. We show that game theory can be used successfully to interpret the equilibrium-selecting behavior observed in computational experiments with adaptive bargaining agents. Agreement between theory and experiment is especially good when the agents experience an intermediate time pressure. Deviations from classical game theory are, however, observed in several experiments. Violent nonlinear oscillations may for instance occur in the single-stage ultimatum game. We demonstrate that the specific evolutionary model governing agent selection is an important factor under these conditions. In multiple-stage games, the evolving agents do not always fully perceive and exploit the finite horizon of the game (even when time pressure is weak). This effect can be attributed to the boundedly-rational behavior of the adapting agents. Furthermore, when the agents discount their payoffs at a different rate, the agent with the largest discount factor is not able to exploit his bargaining power completely, being under pressure by his impatient opponent to reach an early agreement. Negotiations over multiple issues, a particularly important aspect of electronic trading, are studied in a companion paper cite{Gerding:00. We are currently investigating the behavior of more complex and powerful bargaining agents

Stabilization of tag-mediated interaction by sexual reproduction in an evolutionary agent system

The evolution of cooperation in a system of agents playing the iterated prisoner's dilemma (IPD) is investigated. We present results for the standard two-person IPD as well as the more general N-person IPD (NIPD) game. In our computational model, agents can recognize each other and decide whether to interact or not, based upon ``tags'' (labels). We consider the evolutionary stability of the evolving populations. Previous work is extended by introducing sexual reproduction (recombination) of agents and by analyzing its influence on the evolving populations. We observed the occasional formation of very stable cooperative societies, as opposed to previous results without sexual reproduction. These cooperative societies are able to resist invasions of ``mimics''(defecting agents with the tag of a cooperating agent)

An algorithm for on-line price discrimination

The combination of on-line dynamic pricing with price discrimination can be very beneficial for firms operating on the Internet. We therefore develop an on-line dynamic pricing algorithm that can adjust the price schedule for a good or service on behalf of a firm. This algorithm (a multi-variable derivative follower with adaptive step-sizes) is able to respond very quickly to changes in customers' demand. An additional advantage of the developed algorithm is that it does not require information about individual customers. Given the growing concern about customers' privacy this can be of great practical importance. Computational experiments (with different customer behavior models) indicate that our algorithm is able to successfully exploit the potential benefits of on-line price discrimination